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ABSTRACT 

An  efficient  implementation  of  Ada  task  termination  for  shared  memory  MIMD 
architectures  is  presented.  The  solution  is  highly  parallel  and  distributed; 
termination  is  not  synchronized  by  a  centralized  supervisor,  and  there  are  no 
critical  sections.  The  handling  of  block  and  subprogram  termination  that  depend  on 
nested  tasks  is  also  included.  This  implementation  can  be  easily  adapted  to  message 
based  systems. 

1.    Introduction 

An  efficient  implementation  of  Ada  tasking  in  a  distributed  environment  must  contend 
with  several  potentially  expensive  serialization  points,  that  is  to  say,  constructs  whose 
execution  may  take  a  time  proportional  to  the  number  of  tasks  present.  Such  serialization 
points  negate  the  efficiency  that  can  otherwise  be  obtained  by  multitasking.  In  proposed 
parallel  architectures  with  large  numbers  of  processors  ([GLR83],  [Pfi85]),  where  gains 
from  parallelization  will  be  substantial,  it  is  particularly  important  to  minimize  serial 
bottlenecks.  In  this  paper  we  examine  one  such  synchronization  problem,  the  efficient 
implementation  of  the  termination  rules  for  Ada  tasks. 

Task  termination  imposes  a  synchronization  point  on  several  language  constructs.  In 
accord  with  the  block  structure  discipline  of  Ada,  the  end  of  a  block,  subprogram,  or  task 
in  which  tasks  have  been  declared  is  a  synchronization  point:  execution  of  the  construct's 
outer  context  may  not  proceed  until  all  such  tasks  have  terminated.  Termination  is 
complicated  by  the  terminate  alternative  feature  of  select  statements.  Because  of  this 
feature,  sets  of  tasks  must  be  terminated  together  based  on  the  state  of  all  of  the  tasks  in 
the  set.  Therefore,  many  implementations  in  a  single  processor  environment  rely  on  a 
centralized  supervisor  ([Ros83],  [RB84])  to  detect  terminatability  and  to  lock  out  other 
tasks  during  task  termination  synchronization.  More  complex  solutions  to  task  termination 
have  been  presented  for  distributed,  nonshared  memory  machines  [Kaf85],  [FW86].  The 
solution  in  [Kaf85]  involves  message  passing  among  sets  of  tasks  to  achieve  the  effects  of  a 
centralized  supervisor;  for  example,  a  task  must  sometimes  exchange  messages  with  its 
parent  to  determine  whether  to  engage  in  a  rendezvous.  Related  work,  not  directly 
involving  Ada,  is  presented  in  [CL82].  An  algorithm  is  given  for  distributed  termination  in 
systems  with  dynamic  creation  of  tasks,  with  the  restriction  that  tasks  must  know  with 
whom  they  can  communicate  when  they  are  created. 


Little  has  been  published  describing  task  termination  on  MIMD,  shared  memory 
machines,  an  architecture  for  which  Ada  tasking  is  well  suited  ([SS85]).  The  solution 
presented  here  is  for  a  shared  memory  architecture  (in  particular,  the  Ultracomputer 
[GLR83]),  but  can  be  readily  adapted  to  simplify  the  message  based  approach  as  well.  It 
generalizes  the  single  processor  approach  described  in  [RB84]  to  a  multiprocessor 
environment  by  eliminating  the  centralized  supervisor  and  distributing  its  work  among  the 
synchronizing  tasks.  We  prove  the  correctness  of  our  implementation  by  showing  that  the 
critical  sections  provided  by  a  centralized  supervisor  are  not  required.  We  neither  employ 
a  complex  locking  mechanism  nor  rely  on  extensive  communication  among  tasks.  Task 
synchronization  is  for  the  most  part  local,  i.e.  between  task  and  subtask.  Finally,  the 
implementation  presented  is  complete  in  that  it  includes  the  handling  of  block  and 
subprogram  termination  which  depend  on  nested  task  termination. 

The  simplicity  of  the  distributed  solution  is  surprising  in  view  of  the  richness  of  Ada 
tasking  features.  The  inclusion  of  task  types  and  access  to  task  types  in  ANSI/TvIIL-STD- 
1815  Ada,  while  providing  an  orthogonal  and  powerful  language  construct,  seemingly 
complicates  questions  of  distributed  termination.  For  example,  the  claim  in  [Cle82],  that 
only  the  parent  and  siblings  of  a  task  waiting  at  a  terminate  alternative  are  able  to  call  one 
of  its  entries,  does  not  hold  in  ANSI/Ada.  Because  of  the  existence  of  access  task  types, 
tasks  can  be  dynamically  created  using  an  allocator,  and  pointers  to  tasks  can  be  freely 
assigned  to  variables  and  transmitted  as  actual  parameters  in  calls.  Fortunately,  Ada 
termination  rules,  described  in  the  next  section,  allow  a  straightforward  distributed 
implementation  in  spite  of  these  complications. 

2.   Ada  Task  Termination  Rules 

The  full  description  of  Ada  task  termination  rules  is  given  in  [LRM83]  9.4.  We  give  a 
brief  summary  of  these  rules  here,  and  illustrate  them  with  a  few  examples. 

Tasks  are  directly  dependent  on  master  constructs,  where  a  master  construct  m  is  either 
another  task,  a  block,  a  subprogram,  or  a  library  package.  To  precisely  define  direct 
dependency,  we  must  consider  two  cases: 

(1)     a  task  created  by  the  evaluation  of  an  allocator  depends  directly  on  the  master  that 
elaborates  the  corresponding  access  type  definition; 
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(2)    otherwise,  a  task  depends  directly  on  the  master  whose  execution  creates  the  task  object. 

If  the  direct  master  of  a  task  r  is  a  block,  then  t  is  said  to  be  an  inner  dependent  of  the 
task  that  is  executing  the  block.  Such  a  task,  in  effect,  has  two  masters:  its  direct  master, 
which  is  a  block,  will  be  referred  to  as  its  master  block;  an  indirect  master,  which  is  the 
task  executing  the  block,  will  be  referred  to  as  its  master  task.  This  distinction  is  essential 
to  the  algorithms  presented  below.  If  the  direct  master  of  a  task  t  is  another  task,  then  t 
has  a  master  task,  but  no  master  block.  If  the  direct  master  of  r  is  a  subprogram,  then  the 
subprogram  will  also  be  referred  to  as  the  master  block  of  t.  In  this  case,  t  has  a  master 
block,  but  no  master  task. 

The  notion  of  dependence  is  generalized  as  follows:  a  task  t  depends  on  a  master 
construct  m  if  it  directly  depends  on  m,  is  an  inner  dependent  of  m,  or  depends  on  another 
master  m'  that  depends  on  m. 

Task  termination  then  takes  place  under  the  following  conditions.  If  a  task  has  no 
dependent  tasks,  it  terminates  when  it  has  completed,  i.e.  when  it  has  finished  executing  its 
statements.  If  a  task  has  dependent  tasks,  it  terminates  when  it  has  completed,  and  all  of 
its  dependents  have  terminated. 

A  block  or  subprogram  that  has  dependent  tasks  can  only  be  left  when  all  of  its 
dependents  have  terminated.  For  consistency  we  will  say  that  a  block  or  subprogram 
terminates  when  it  is  left. 

This  straightforward  rule  is  complicated  by  the  Ada  terminate  alternative,  which 
provides  another  way  in  which  a  task  may  terminate.  A  task  terminates  when  its  e.xecution 
has  reached  an  open  terminate  alternative  in  a  select  statement,  and  the  following  conditions 
are  satisfied: 

(1)  The  task  depends  on  some  master  whose  execution  is  completed  (hence  not  a  library- 
package). 

(2)  Each  task  that  depends  on  the  master  considered  is  either  already  terminated  or  similarly 
waiting  on  an  open  terminate  alternative  of  a  select  statement. 

When  both  conditions  are  satisfied,  the  task  considered  becomes  terminated,  together  with 
all  tasks  that  depend  on  its  master.  We  call  the  termination  of  a  task  and  all  of  its 
dependents  the  termination  wave  of  the  task.  Tasks  that  depend  on  library  packages  are  a 
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special  case,  and  are  discussed  in  the  Appendix. 

Some  of  the  imphcations  of  these  rules  are  demonstrated  in  the  following  examples.  It 
is  useful  to  represent  the  dependency  relation  among  master  tasks  and  dependent  tasks  by 
trees.  The  root  node  of  a  dependency  tree  is  a  master  task  or  a  master  block;  the  children 
of  a  tree  node  are  all  the  tasks  that  depend  on  the  node.  (A  master  block  cannot  be  an 
interior  node  in  the  dependency  trees,  since  a  master  block  is  not  a  dependent  of  any  other 
construct.  A  task  t  that  has  both  a  master  task  and  a  master  block  is  an  interior  node  in  two 
trees.)  After  each  example,  we  give  its  dependency  tree(s). 

Example  1  -  Tasks  With  Terminate  Alternatives 

This  example  illustrates  nested  tasks  with  terminate  alternatives  in  select  statements. 
A  portion  of  the  dependency  tree  from  Example  1  is  shown  in  Figure  1.  If  all  of  the  tasks 
in  the  tree  are  executing  their  select  loops,  termination  can  take  place  only  if  all  of  the  child 
and  grandchild  tasks  are  waiting  at  terminate  alternatives.  All  of  the  tasks  in  the  tree  will 
terminate  together  in  a  single  termination  wave.  Note  that  a  given  child  task  may  be  called 
by  any  of  the  15  grandchild  tasks,  and  that  even  if  all  of  the  child  tasks  are  waiting  at 
terminate  alternatives,  the  state  of  a  child  may  change  because  another  task  (e.g.  a 
grandchild)    may  invoke    it,  and  so  termination  may  not  be  possible. 


master 


dependence: 

entry  call: 


Figure  1 
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task  master; 

task  body  master  is 


task  type  children  is 

entry  el(i:  integer); 
end  children; 

child:  array  (1..5)  of  children; 
—  Other  declarations. 

task  body  children  is 

task  type  grandchildren  is 

entry  e2(j:  integer); 
end  grandchildren; 

grandchild:  array  (1..3)  of  grandchildren; 
—  Other  declarations. 

task  body  grandchildren  is 
begin  —  Code  executed  by  grandchild. 
loop 

select 

accept  e2(j:  integer)  do 

child(j).el(j); 

end  e2; 

or 

terminate; 
end  select; 
end  loop; 
end  grandchildren; 
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begin    —  Code  executed  by  child. 
loop 

select 

accept  el(i:  integer)  do 


end  el; 


or 


terminate; 
end  select; 
end  loop; 
end  children; 


begin 


—  Code  body  of  master. 
end  master; 

Example  2  -   Tasks  That  Exchange  Tasks  As  Entry  Parameters 

A  task  can  also  be  called  by  a  task  that  is  not  a  dependent  of  its  master,  as  illustrated 
below.    The  dependency  tree  for  the  program  is  shown  in  Figure  2. 


caller 


master 


middleman 


^1,        called 


Figure  2 
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task  master; 

task  body  master  is 


task  middleman  is 

entry  el(i:  integer); 
end  middleman; 

task  type  called_type  is 

entry  e2(j:  integer); 
end  called_type; 

task  caller  is 

entry  e(tasktocall:  in  ca]led_type); 
end  caller; 

task  body  called_type  is 
begin 

loop 

select 

accept  e2(j:  integer)  do 

end  e2; 

or 

terminate; 
end  select; 
end  loop; 
end  called_type; 

task  body  middleman  is 

called  :  called_type;  —  Create  dependent  task. 
begin 

caller. e(called);  —  Pass  dependent  task  to  caller. 
loop 

select 

accept  el(i:  integer)  do 

end  el; 

or 

terminate; 
end  select; 
end  loop; 

end  middleman; 


task  body  caller  is 
begin 
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accept  e(tasktocall:  in  called_type)  do 
tasktocall.e2(17); 

end  e; 

end  caller; 

--  Code  body  of  master. 

end  master; 

Example  3  -  Task  With  Allocators 

This  example  illustrates  dynamic  task  allocation.  The  dependency  tree  is  shown  in 
Figure  3.  After  the  task  dependent  is  activated,  it  dynamically  creates  tasks  tl  and  t2. 
These  dynamically  created  tasks  are  dependents  of  dependent  because  their  access  type  is 
defined  in  dependent.  Additionally,  every  time  the  entry  e  is  called  and  the  rendezvous  is 
complete  (e.g.  see  the  body  of  master),  dependent  creates  a  new  task  t3 ,  also  a  direct 
dependent  of  dependent.  Termination  can  take  place  when  dependent  is  waiting  at  a 
terminate  alternative,  all  tasks  created  by  dynamic  allocators  have  terminated,  and  master 
is  complete.  Note  that,  depending  on  how  many  times  the  entry  e  is  called,  an  arbitrary 
number  of  subtasks  t3  may  have  been  generated. 


tl 


t2  13        t3 


o 
t3 


Figure  3 
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task  master; 

task  body  master  is 


task  dependent  is 

entry  e(i:  integer); 
end  dependent; 

task  type  dynamic; 
task  body  dynamic  is 
begin 

end  dynamic; 

task  body  dependent  is 

type  dynamic_ptr  is  access  dynamic; 
tl,  t2,  t3:  dynamic_ptr; 
begin 

tl  :=  new  dynamic; 
t2  :=  new  dynamic; 
loop 

select 

accept  e(i:  integer)  do 

t3  :=  new  dynamic; 

end  e; 
or 

terminate; 
end  select; 
end  loop; 
end  dependent; 


begin  —  Body  of  master. 

dependent. e(7); 
end  master; 

Example  4  -  Master  Blocks 

This  example  illustrates  tasks  dependent  on  master  blocks,  and  some  of  the 
complications  that  derive  from  dependency  rules.  Figure  4  shows  the  dependency  trees  of 
the  master  task  and  the  master  block.  The  declaration  of  task  tl  is  elaborated  by  task 
master,  so  tl  is  a  direct  dependent  of  master.    Master  contains  an  inner  block:  since  the 
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master 


inner 


o 
t2 


Figure4 


dependence: 
entry  call: 

innerdependence 


declaration  of  task  t2  is  elaborated  in  the  block,  the  block  is  the  direct  master  of  t2.  Task 
t2  is  an  inner  dependent  of  master.  Task  t3  is  also  created  via  an  allocator  within  the  inner 
block.  However,  since  the  access  type  of  ri  is  declared  in  master,  (see  rule  (1),  Section  2), 
t3  is  the  direct  dependent  of  master. 

The  inner  block  can  exit  when  its  dependent  t2  terminates  or  waits  on  a  terminate 
alternative  and  the  block  completes.  Note,  however,  that  the  block  itself  contains  a  select 
loop  with  a  terminate  alternative.  If  the  select  loop  is  executed,  it  is  possible  that  the  inner 
block  may  never  complete.  In  this  case  the  task  master  (not  the  block)  is  waiting  to 
terminate:  master  may  then  terminate  according  to  the  rules  defined  above,  and  when  the 
direct  and  inner  dependents  of  master,  namely  tl ,  t2,  and  t3 ,  have  terminated  or  are 
waiting  to  terminate.  For  this  reason,  task  t2  is  in  the  dependency  trees  of  both  master  and 
the  inner  block. 
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task  master  is 

entry  em; 
end  master; 

task  body  master  is 

task  type  dependent  is 

entry  e; 
end  dependent; 

type   dependent_ptr  is  access  dependent; 

task  body  dependent  is 
begin 

loop 

select 

accept  e  do 

end  e; 
or 

terminate; 
end  select; 
end  loop; 
end  dependent; 

tl  :  dependent;   —  direct  dependent  of  master 

begin  —  body  of  master. 

inner:   declare 

t2  :  dependent;  —  direct  dependent  of  inner. 
t3  :  dependent_ptr; 
begin 

t3  :—  new  dependent;  --  direct  dependent  of  master. 

loop 

select 

accept  em  do 

end  em; 
or 

terminate;  —  Master. 
end  select; 
end  loop; 

end  inner; 
end  master; 
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3.   A  Necessary  And  Sufficient  Condition  For  Termination 

We  introduce  the  notion  of  quiescence ,  and  then  prove  a  condition  on  dependency 
trees  that  enables  a  distributed  implementation  of  termination. 

Definition:  If  r  is  a  task  with  no  dependents,  then  t  is  quiescent  if  t  is  complete  or  if  t 
is  waiting  at  a  terminate  alternative.  If  t  has  dependents,  then  t  is  quiescent  if  t  is  complete 
or  if  f  is  waiting  on  a  terminate  alternative,  and  all  dependents  of  t  are  quiescent.  A  task 
which  is  neither  quiescent  nor  terminated  is  said  to  be  active.  Finally,  if  i  is  a  master  block 
(i.e.  a  block  or  subprogram  with  dependent  tasks),  then  b  is  quiescent  if  b  is  complete  and 
all  dependents  of  b  are  quiescent. 

Note  that  if  a  task  f  is  quiescent  and  waiting  on  a  terminate  alternative,  it  is  not  valid 
to  terminate  it,  since,  for  example,  its  master  may  not  be  complete  and  may  still  call  t.  We 
can,  however,  make  the  following  claim. 

Proposition  1.  If  a  master  m  is  quiescent  and  complete  (i.e.  not  waiting  at  a  terminate 
alternative),  then  m  may  be  terminated. 

Proof.  Because  of  the  recursive  definition  of  quiescence,  any  task  that  depends  on  m  is 
either  complete  or  waiting  at  a  terminate  alternative.  Thus  m  may  be  terminated  according 
to  [LRM83]9.4. 

The  following  proposition  allows  us  to  implement  termination  in  a  simple  distributed 
fashion. 

Proposition  2.  While  a  master  m  is  quiescent,  no  dependents  of  m  can  be  called  and 
no  new  dependents  of  nz  can  be  created. 

Therefore,  the  only  way  for  m  to  become  active  (assuming  m  is  waiting  on  a  terminate 
alternative)  is  for  another  task  to  invoke  m  directly.  The  implication  of  Proposition  2  is  that 
once  a  termination  wave  has  begun,  no  task  from  outside  the  wave  may  call  a  task  being 
terminated  in  the  wave.  Thus  it  is  not  necessary  for  the  tasking  system  to  lock  out  other 
tasks  during  a  termination  wave,  i.e.  for  a  termination  wave  to  be  implemented  as  an 
indivisible  operation.  Furthermore,  the  bookkeeping  needed  to  check  for  and  initiate 
termination  can  be  performed  in  a  relatively  local  manner,  as  described  in  Section  4.  We 
first  give  a  proof  of  Proposition  2. 
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Proof.  We  will  prove  the  proposition  in  two  parts:  first  the  absence  of  callers  and 
then  the  absence  of  new  dependents. 

No  Dependents  Can  Be  Called.  Let  r  be  a  dependent  of  a  quiescent  master  m,  and 
suppose  there  is  another  task  tl  that  invokes  t.  Then  tl  cannot  be  a  dependent  of  m,  since 
all  dependents  of  m  are  quiescent.  First,  assume  that  r  is  a  task  object.  For  tl  to  call  r,  it 
must  be  the  case  that  m,  or  a  dependent  of  m  that  can  access  f,  passes  /  to  tl  through  a 
sequence  of  one  or  more  entry  calls  to  a  task  that  is  not  in  the  dependency  tree  rooted  at 
m.  (If  m  is  the  direct  master,  t  is  declared  in  m,  and  is  not  otherwise  accessible  by  name 
outside  the  dependency  tree  of  m.  If  m  is  not  the  direct  master,  r  is  declared  in  an  inner 
block  or  dependent  task  of  m.  All  dependents  of  the  direct  master  of  t  are  also  dependents 
of  m,  and  so  again  t  is  not  accessible  by  name  outside  the  dependency  tree  of  m.)  Let  o  be 
the  task  that  depends  on  m  and  passes  t  to  an  outer  dependency  tree.  Task  objects  are  of 
limited  type;  therefore,  t  may  not  be  passed  by  assignment  (e.g.  to  a  global  variable).  For 
tl  to  be  able  to  reference  r  (passed  as  an  in  parameter),  it  must  therefore  be  the  case  that  o 
is  currently  executing  an  entry  call,  and  so  cannot  be  quiescent  (See  Figure  5).  This  is  a 
contradiction. 


Figure  5 
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Next,  assume  r  was  created  by  the  evaluation  of  an  allocator.  Task  pointers  are  not 
limited  and  can  be  assigned.  Hence  a  task  tl  that  receives  a  pointer  to  r  as  an  in  parameter 
can  call  r  even  after  the  rendezvous  is  completed.  However,  we  claim  that  if  a  task  tl  calls 
t,  tl  must  be  a  dependent  of  its  direct  master.  This  follows  from  the  visibility  rules  of  Ada. 
By  definition,  if  m'  is  the  direct  master  of  f,  its  access  type  declaration  appears  in  m' .  For 
tl  to  call  f  it  must  be  able  to  see  the  access  type  declaration,  and  therefore  the  type  of  t] 
must  be  declared  within  the  scope  of  the  direct  master  of  f.  It  follows  that  tl  is  a 
dependent  of  m,  and  must  be  quiescent.  Therefore  no  dependents  of  m  can  be  called  while 
m  is  quiescent. 

No  Dependents  Can  Be  Created.  Suppose  that  a  task  t]  creates  a  task  f,  which 
becomes  a  dependent  of  a  quiescent  master  m.  As  above,  the  corresponding  access  type  is 
declared  in  the  direct  master  of  r,  and  the  access  type  definition  must  be  visible  to  tl . 
Thus,  t]  must  be  a  dependent  of  the  direct  master  of  t,  and  hence  be  a  dependent  of  m.  By 
hypothesis,  such  a  task  must  be  quiescent,  and  therefore  cannot  be  executing  an  allocator. 
Thus  while  m  is  quiescent,  no  new  dependent  can  be  created.  This  completes  the  proof  of 
Proposition  2. 

4.    Implementation  Of  Termination  On  An  MIMD  Architecture. 

To  implement  distributed  termination,  we  associate  a  counter  no_term  with  each 
master  to  keep  track  of  the  number  of  its  direct  (and  inner)  dependents  that  are  not 
quiescent.  It  is  clear  from  the  definition  of  quiescence,  that  if  all  direct  (and  inner)  tasks 
are  quiescent,  then  all  dependents  are  also  quiescent. 

For  a  master  task  no_term  is  defined  as; 

no_term(t)  =  if  f  is  complete  or  waiting  on  a  terminate  alternative  then 
the  number  of  active  direct  and  inner  dependents  of  f 
else 

the  number  of  active  direct  and  inner  dependents  of  r  +  1. 

Thus  no_term  for  a  task  includes  the  task  itself  in  the  count  of  active  dependents.  The  extra 
1  in  no_term  when  a  master  is  not  complete  or  waiting  on  a  terminate  alternative  allows  us 
to  implement  the  test  for  quiescence  as  an  indivisible  operation.  Inner  dependents  are  also 
counted,  to  handle  the  case  illustrated  by  Example  4,  Section  2.  For  a  master  block  no_term 
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is  defined  as: 

no_term(b)  =  if  Z?  is  complete  then 

the  number  of  active  direct  dependents  of  b 
else 

the  number  of  active  direct  dependents  of  b  +  1. 

It  follows   from  Proposition    1,   for  a  master  m,  that  when   no_term(m)   =    0,  m  may  be 
terminated. 

The  control  block  of  each  task  contains,  in  addition  to  a  no_term  counter,  pointers  to 
its  master  task  and  master  block,  and  a  status  field.  The  possible  values  of  status,  used 
below,  are  self-explanatory.  Each  block  or  subprogram  which  is  a  master  construct  also 
contains  a  no_term  counter.  An  additional  fl&g  first,  initially  0,  is  also  needed  in  a  master 
task  control  block  to  ensure  that  only  a  single  dependent  task  will  notify  the  master  task  to 
terminate. 

The  next  subsections  present  the  detailed  implementation  of  termination.  The  only 
primitive  indivisible  instructions  used  below  are  simple  (but  unconventional)  increment  and 
decrement  instructions.  We  will  use  inc(v)  and  dec(v)  to  designate  them.  The  operation 
inc(v)  (resp.  dev(v))  is  an  indivisible  instruction  which  simultaneously  increments  (resp. 
decrements)  v  by  1  and  returns  the  value  of  v  before  the  operation.  This  somewhat 
unusual  choice  of  primitives  is  influenced  by  the  existence  of  the  atomic  fetch-and-add 
operation  on  the  Ultracomputer  [GLR83],  but  can  be  implemented  on  other  architectures 
as  well.  The  advantage  of  the  fetch-and-add  operation  is  that  it  implements  inc  and  dec  in 
such  a  way  that  the  time  for  many  processors  to  simultaneously  perform  an  inc{v)  or  dec(y) 
operation  on  the  same  variable  v  is  the  same  as  for  a  single  processor.  (Thus  serial 
bottlenecks  that  might  otherwise  be  incurred  from  the  inc  and  dec  operations  used  below 
are  eliminated  on  an  Ultracomputer.) 

In  addition,  the  following  code  assumes  the  existence  of  a  (distributed)  scheduler  that 
manages  various  queues.  The  only  actions  involving  the  scheduler  below  are  calls  to  block 
and  unblock,  which  have  the  usual  semantics.  In  our  implementation  unblock  has  two 
parameters:  the  task  to  be  unblocked  and  the  event  that  caused  the  task  to  become 
unblocked. 


UltracompuUr  Note  107  Page  16 


4.1.   The  Distributed  Termination  of  Blocks  and  Subprograms 

In  this  section  we  deal  only  with  the  termination  of  master  blocks  and  subprograms, 
that  is,  blocks  and  subprograms  that  have  dependent  tasks.  Because  we  are  only  dealing 
with  master  blocks,  the  description  of  some  of  the  actions  performed  when  tasks  are 
activated,  become  quiescent,  etc.,  is  omitted.  Termination  of  tasks  is  somewhat  more 
complicated,  and  is  described  in  more  detail  in  the  next  section.  The  nojlerm  counter  in 
the  activation  record  of  the  master  block  indicates  when  the  block  can  be  exited;  a  brief 
summary  of  the  actions  performed  on  no_term  is  first  specified,  and  code  to  implement  this 
specification  follows. 

When  a  block  b  is  entered  its  no_term  is  set  to  1.  When  a  task  that  depends  directly 
on  b  is  activated,  it  increments  no_term(b).  Similarly,  when  a  task  directly  dependent  on  b 
accepts  an  entry  call  while  waiting  on  a  terminate  alternative,  and  the  task  was  quiescent 
before  the  call,  it  increments  no_term(b). 

When  a  block  b  is  complete,  its  no_term  is  decremented.  If  nn_term(h)  is  0,  then  h  is 
quiescent  and  can  be  terminated.  When  a  dependent  task  t  becomes  quiescent,  (either  by 
completing  or  waiting  on  a  terminate  alternative,  and  decrementing  its  own  counter  to  0),  it 
decrements  no_ierm(b).  If  r  discovers  that  its  master  block  b  is  now  quiescent,  i.e. 
no_term(b)  is  0,  it  notifies  its  master  task  (which  is  of  course  executing  b)  to  resume  and 
begin  a  termination  wave  for  b. 

Thus  a  master  block  b  can  terminate  when  b  is  complete  and  no_term(b)  becomes  0. 
Before  terminating,  b  tells  all  its  direct  dependents  that  have  not  already  terminated  to 
terminate.  Each  direct  dependent  does  the  same  recursively.  (Notifying  direct  dependents 
to  terminate  is  a  serialization  point  in  the  implementation.  If,  however,  none  of  the  direct 
dependents  are  waiting  on  terminate  alternatives,  all  of  the  direct  dependents  must  already 
be  terminated,  and  there  is  no  serialization.) 

The  code  to  implement  the  termination  of  masters  that  are  blocks  or  subprograms  is 
given  below.    The  variable  5-e// designates  the  task  executing  the  code. 

(1)  When  a  master  block  B  is  entered,  but  before  its  declarations  are  elaborated: 

B.NO_TERM  :=  1;   --  no  active  dependents,  but  not  complete. 

(2)  When  a  master  block  B  completes: 
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if  dec(B.NO_TERM)    /=    1  then 

—  Wait  for  dependents  to  become  quiescent 
block(self); 

end  if; 

-  Upon  awakening,  start  a  terminate  wave 
for  each  direct  dependent  T  of  B  loop 

if  T. STATUS  =  SELECT_TERM  then 
unblock(T,TERMINATE); 

end  if; 
end  loop; 
~  Exit  from  block. 

(3)  When  a  task,  self,  becomes  quiescent: 

if  dec(self.MASTER_BLOCK.NO_TERM)  =  1  then 

—  The  master  is  quiescent. 

—  Awaken  master  task  to  start  termination  wave  of  inner  block. 
unblock(self.MASTER_TASK,  BLOCK_TERMINATION_WAVE); 

end  if; 

(4)  When  a  task  becomes  active: 

inc(self.MASTER_BLOCK.NO_TERM); 

4.2.   The  Distributed  Termination  of  Master  Tasks 

The  operations  involved  in  task  termination  are  similar  to  those  for  block  and 
subprogram  termination.  There  are,  however,  more  complications.  Once  a  block  or 
subprogram  becomes  quiescent  it  cannot  become  active  again.  In  contrast,  a  quiescent  task 
that  is  waiting  on  a  terminate  alternative  may  become  active  by  being  called  by  a  master, 
sibling,  or  sibling  dependent.  Also,  unlike  a  block,  a  task  has  masters  that  must  be 
updated  when  the  status  of  a  task  changes.  The  no_term  counter  of  a  task  t  is  incremented 
as  follows: 

(i)     When    r  is  activated,  it  sets  its  no_term  to  1. 
(ii)    If  t   is   waiting   on    a   terminate   alternative   and   enters   a   rendezvous,   it   increments 

no_term . 
(iii)  Finally,  no_term(t)  is  incremented  by  a  direct  or  inner  dependent  when  the  dependent 

is  activated  or  when  the  dependent  changes  from  quiescent  to  active. 

A  significant  observation  is  that  when  a  quiescent  task  becomes  active  (or  a  task  is 
created),  it  is  only  necessary  to  propagate  the  information  one  level  up  a  dependency  tree  - 
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i.e  to  its  master  task  (and/or  master  block).  This  follows  from  Proposition  2:  if  a  task  r  is 
invoked,  it  must  be  the  case  that  its  master  task  is  not  quiescent,  and  hence  no  change  of 
state  (from  quiescent  to  active)  can  occur  in  the  master  task.  Consequently,  no  further 
updating  nor  checking  need  be  performed.  A  similar  remark  applies  when  a  task  is 
dynamically  created. 

Counters  are  decremented  as  follows.  When  t  completes  or  waits  on  a  terminate 
alternative,  it  decrements  its  no_term.  If  in  so  doing  it  becomes  quiescent,  it  also 
decrements  nojerm  of  its  master  task.  When  a  task  t  discovers  that  its  master  task  m  is 
now  quiescent  and  m  is  complete,  then  r  notifies  m  to  begin  a  termination  wave;  if  m  is 
waiting  on  a  terminate  alternative,  then  recursively  t  decrements  masters  of  its  master  task 
until  either  of  the  following  occurs: 

(1)  r  finds  a  master  task  that  is  not  quiescent  (in  which  case  t  either  blocks    if  it  is  not 
complete  or  terminates  its  own  dependents  if  it  is  complete), 

(2)  r   finds    a    master    task    that    is    quiescent    and    complete,    and    can    therefore    start   a 
downward  termination  wave. 

This  recursive  decrementing  of  nojterm  and  checking  for  quiescence  is  the  only  time 
that  information  must  be  propagated  for  more  than  one  level  in  a  dependency  tree.  It  need 
not,  however,  be  an  indivisible  operation,  nor  be  performed  in  a  critical  section. 
Furthermore,  propagation  and  checking  need  only  be  performed  up  a  single  branch  in  the 
dependency  tree.  No  subsequent  rechecking  down  the  tree  is  required.  These 
observations  follow  from  Proposition  2:  while  a  task  t  is  updating  an  ancestor  m  and  m  is 
quiescent,  the  status  of  all  the  dependents  of  m  cannot  change. 

However,  a  termination  wave  is  a  complex  affair:  the  task  that  starts  the  wave  (the 
master  task  awakened  by  step  2  above)  must  both  recursively  update  its  masters'  nojterms 
since  it  has  become  quiescent,  and  notify  its  dependents  to  terminate. 

Implementation  details  follow.  The  code  presented  is  for  tasks  that  have  both  master 
blocks  and  master  tasks.    If  a  task  has  no  master  block,  the  code  is  simpler. 
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(1)  When  a  task  T  is  activated,  prior  to  activating  any  tasks  declared  in  T: 

self.NO.TERM  :=  1; 

inc(self.MASTER_TASK.NO_TERM); 

inc(self.MASTER_BLOCK.NO_TERM); 

(2)  Upon  entering  a  rendezvous  in  a  selective  wait  with  a  terminate  alternative: 

if  inc(self.NO_TERM)  =  0  then 

—  Was  quiescent  before  rendezvous. 

--  Restore  masters'  NO_TERM  counts. 
inc(self.MASTER_TASK.NO_TERM); 
inc(self.MASTER_BLOCK.NO_TERM); 
end  if; 

(3)  Upon  reaching  a  selective  wait  with  a  terminate  alternative  and  no  acceptable  calls: 

if  dec(self.NO_TERM)  =  1  then 

—  Am  quiescent  now. 

—  Propagate  information  up  tree. 
CHECK_TERMINATION(self.MASTER_TASK,  self.MASTER3L0CK); 

end  if; 

block(self);   —  Awakened  by  either  a  rendezvous  or  to  terminate. 

(4)  When  a  task  completes: 

self.STATUS  :=  COMPLETE; 
if  dec(self.NO_TERM)  /=  1  then 

—  Wait  for  dependents  to  be  quiescent. 
block(selO; 

end  if; 

—  Am  quiescent  now. 

--  Update  and  check  master(s). 
CHECK_TERMINATION(self.MASTER_TASK,  self.MASTER_BLOCK); 

—  Notify  dependents  to  terminate. 

for  each  direct  and  inner  dependent  T  of  self  loop 

if  T. STATUS  =  SELECT_TERM    then 
unblock(T,TERMINATE); 

end  if; 
end  loop; 

—  Terminate  self. 

(5)  Recursive  subprogram  to  update  and  check  master  tasks  and  block. 


UltracompnUr  Note  107  Page  20 


procedure  CHECK_TERMINATION(M,  B)  is 

—  called  by  a  quiescent  dependent  of  M  and  B. 
begin 

if  M  /=  null  and  then  dec(M.NO_TERM)  =  1  then 
--  Master  task  is  quiescent, 
if  M. STATUS  =  COMPLETE  then 

--  Must  check  quiescence  again  to 

—  avoid  potential  race  condition. 

ifM.NO_TERM    =  0  and  then 

—  M  is  quiescent  and  complete. 

--  Check  for  first  caller.^ 
inc(M. FIRST)  =  0  then 

~  Start  termination  wave. 
unblock(M,  TERMINATION_WAVE); 
end  if; 
else 

--  M  is  waiting  on  terminate  alternative. 

—  Check  M's  master. 
CHECK_TERMINATION(M.MASTER_TASK, 

M.MASTER3LOCK); 
end  if; 
end  if; 

—  If  the  direct  master  of  self  is  a  block  B,  update  and  check  B. 
if  B  /=  null  and  then    dec(B.NO_TERM)  =  1  then 

--    B  is  quiescent. 
—    (see  Section  4.1) 

unblock(M,  TERMINATION.WAVE);  --  M  is  executing  B 
end  if; 
end  CHECK_TERMINATION; 


^A  race  condition  is  possible  here  because  the  check  for  quiescence  and  completion  is  not  an  indivisible 
operation.  After  the  first  test  for  quiescence  but  before  M. STATUS  is  checked,  M  may  be  invoked  and  a 
dependent  Tl  of  M  activated.  If  M  subsequently  completes,  the  second  test  (for  completeness)  will  succeed, 
even  though  M  is  no  longer  quiescent.  Therefore,  an  additional  test  for  quiescence  is  necessary.  See  Section 
4.3. 

'  Only  the  first  dependent  to  discover  tcrminatability  should  start  the  termination  wave.  In  a  similar 
scenario  as  above,  it  is  possible  for  more  than  one  dependent  of  M  to  discover  that  M  is  quiescent  and  complete. 
To  see  how  this  can  occur,  consider  Example  3  in  Section  2.  Suppose  task  i2  completes,  finds  its  master 
dependent  waiting  to  terminate,  and  then  decrements  the  counter  of  master.  Suppose  master  is  waiting  on  a 
terminate  alternative,  and  that  its  counter  becomes  0.  Then,  another  task  awakens  master,  which  then  calls 
entry  e  of  dependent,  and  dependent  creates  a  new  task  t3.  Next  master  completes,  t3  completes,  finds 
dependent  quiescent,  and  updates  the  counter  of  master.  If  t2  has  done  nothing  in  the  meantime,  both  t2  and  t3 
will  find  master  complete  and  quiescent,  and  hence,  signal  master  to  start  a  termination  wave. 
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4.3.   Absence  of  Race  Conditions 

We  must  show  that  there  is  no  premature  termination  or  failure  to  detect  when  the 
conditions  for  termination  are  met  due  to  race  conditions.  The  potential  for  race  conditions 
exist  because  more  than  one  dependent  task  may  access  a  task  control  block  concurrently, 
and  some  of  the  actions  involved  in  bookkeeping  are  not  indivisible. 

As  an  example  of  a  potentially  incorrect  state  (whose  existence  we  must  rule  out),  for 
a  master  m,  no_term(m)  might  become  0  before  a  dependent  has  a  chance  to  increment  it, 
resulting  in  a  premature  termination  wave.  In  particular,  this  situation  might  possibly  arise 
from  code  fragments  (1)  and  (2)  in  Section  4.2.  Suppose  a  quiescent  task  r  with  master  m 
is  called  and  enters  a  rendezvous.  Then  before  t  has  a  chance  to  increment  no_ierm(m)  all 
of  the  other  dependents  of  m  may  become  quiescent  -  the  last  dependent  of  m  will  then 
initiate  a  termination  wave  prematurely. 

The  following  proposition  is  useful  in  showing  that  such  a  situation  cannot  arise. 

Proposition  3.  While  a  task  r  is  executing  code  fragments  (1)  or  (2)  (Section  4.2), 
nojterm  of  its  master  task  m  (and/or  master  block)  cannot  become  0. 

Proof.  If  m  is  not  waiting  on  a  terminate  alternative  and  not  complete,  no_term(m)  is 
not  0.    Therefore,  assume  m  is  waiting  on  a  terminate  alternative  or  is  complete. 

a)  If  t  is  executing  code  fragment  (1),  then  it  is  in  the  process  of  being  created 
dynamically  by  another  task  tJ  that  is  executing  an  allocator.  (Otherwise  m  would  not 
be  complete  or  waiting  on  a  terminate  alternative,  but  rather  activating  ?.) 

b)  If  f  is  executing  code  fragment  (2),  then  r  is  being  awakened  from  a  quiescent  state  by 
another  task  tl . 

We  claim  that  there  is  at  least  one  task  tl  other  than  r  that  is  a  dependent  of  m  and  is 
either  executing  an  allocator  or  engaged  in  a  rendezvous.  Either  tl  is  a  dependent  of  m,  in 
which  case  we  let  t2  be  f7,  or,  by  an  argument  similar  to  that  in  the  proof  of  Proposition  2, 
there  is  a  dependent  t2  of  m  that  is  engaged  in  a  rendezvous  and  passing  the  name  of  r  to 
tl .  Since  t2  is  in  a  running  state,  the  no_term  counter  of  its  master  task  ml  must  be  greater 
than  0  (A  task  updates  its  master  upon  entering  a  running  state).  If  ml  is  m,  the  proof  is 
complete.  Otherwise,  we  must  show  that  no  master  of  ml  dependent  on  and  including  m 
can  have  a  no_jerm  of  0. 
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Let  m2  be  the  nearest  master  of  m] ,  such  that  no_term(m2)  is  not  0,  but 
no_term(master(m2))  =  0.  Since  no_term(m2)  is  not  0,  the  only  way  this  can  happen  is  if  m2 
has  just  incremented  its  own  counter,  and  not  yet  incremented  the  counter  for  its  master, 
i.e.  is  executing  code  segments  (1)  and  (2)  in  Section  4.2.  Since  m2  has  dependents  already 
activated,  it  cannot  be  executing  code  fragment  (1).  Similarly,  it  is  not  executing  code 
fragment  (2)  because  it  is  not  quiescent  (m2  has  non-quiescent  dependents).  Therefore, 
no_term(master(m2))  is  not  0,  and  by  induction  on  the  chain  of  master  tasks  from  m2  to  m, 
the  proof  is  complete. 

Another  potential  race  condition  involves  the  checks  that  the  actual  termination 
conditions,  completeness  and  quiescence,  are  met.  These  are  two  separate  checks,  and 
hence  not  an  indivisible  operation.  These  checks  are  performed  when  a  dependent 
decrements  the  counter  of  a  master  task  (in  procedure  CHECK_TERMINATION),  and 
when  a  master  task  completes  and  checks  its  own  counter.  In  the  first  case,  the  dependent 
checks  for  no_term  =  0  twice.  Therefore,  if  the  state  changes  between  the  check  of 
no_term  and  the  check  for  completeness,  the  dependent  will  not  initiate  a  termination  wave 
prematurely.  (Note  that  once  a  task  is  complete  its  status  cannot  change.)  In  the  latter 
case,  the  master  task  sets  its  status  to  complete  before  decrementing  its  no_term  counter. 
Thus,  either  the  master  or  the  dependent  will  discover  correctly  the  case  in  which  a 
termination  wave  can  begin. 

5.    Conclusion 

A  simple  implementation  of  task  termination  in  an  environment  in  which  tasks  can  be 
nested  and  created  dynamically  uses  counters:  the  counter  maintains  the  number  of 
subtasks  of  a  given  task,  and  the  last  subtask  to  decrement  the  counter  initiates 
termination.  We  have  shown  that  this  implementation  strategy  can  be  used  for  Ada  tasks 
in  multiprocessor  environment  with  shared  memory,  in  spite  of  the  complexities  of  the 
Ada  tasking  model. 

Furthermore,  our  distributed  implementation  may  be  adapted  for  an  architecture 
without  shared  memory,  in  which  message  passing  must  be  used.  In  such  an  environment, 
when  a  dependent  task  becomes  quiescent  or  active,  it  must  send  a  message  to  its  master  to 
update  its  no_term  counter.  With  our  implementation,  fewer  messages  need  by  exchanged 
than  in  other  proposed  implementations,  e.g.  [Kaf]. 
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APPENDIX  -  LIBRARY  PACKAGES 

A  task  r  can  gain  visibility  to  a  task  pointer  definition  if  the  task  pointer  type  is 
declared  in  a  library  package  and  r  names  the  library  package  in  a  with  clause.  The  rules  of 
Ada  specify  that  any  library  units  named  within  a  main  program  or  its  subunits  are 
elaborated  before  the  execution  of  the  main  program.  The  rules  for  task  dependencies  state 
that  a  task  created  by  the  evaluation  of  an  allocator  directly  depends  on  the  master  that 
elaborated  the  corresponding  access  type  definition.  Thus,  any  tasks  created  by  the 
evaluation  of  an  allocator  whose  type  definition  appears  in  a  library  package  directly 
depend  on  the  library  package.  As  a  consequence,  such  tasks,  as  direct  dependents  of 
library  packages,  can  be  passed  to  the  direct  dependents  of  the  main  program. 
Furthermore,  new  direct  dependents  of  the  library  package  can  be  created  by  the  direct 
dependents  of  the  main  program.  It  would  seem  that  Proposition  2  is  not  true  for  tasks"  that 
are  dependent  on  library  packages. 

However,  the  reference  manual  states  in  9.4(13),  termination  of  the  main  program 
awaits  termination  of  any  dependent  task  even  if  the  corresponding  task  type  is  declared  in  a 
library  package.  On  the  other  hand,  termination  of  the  main  program  does  not  await 
termination  of  tasks  that  depend  on  library  packages;  the  language  does  not  define  whether 
such  tasks  are  required  to  terminate.  So  the  main  program  need  no  be  concerned  with  the 
termination  of  library  packages  and  their  dependents. 

In  order  to  handle  termination  in  a  uniform  manner,  an  implementation  could  have  an 
outermost  task  on  which  the  main  program  and  library  packages  are  dependent.  The  task 
could  either  wait  for  the  tasks  that  are  dependent  on  the  library  packages  to  terminate  or 
cease  to  exist  when  the  main  program  finished  execution.  Either  solution  adheres  to  the 
reference  manual,  which  does  not  require  that  tasks  that  depend  on  library  packages 
terminate,  and  the  termination  of  the  main  program  is  in  no  way  affected. 
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